The present invention relates to an image predictive decoding method of predictively decoding a compressed digital image and, more particularly, to an overlapped compensation method for interlaced images, an apparatus for performing the method, and a data storage medium for storing the method.
Compressive coding is required to store and transmit a digital image efficiently. There have been proposed methods of compressively coding a digital image, such as DCT (Discrete Cosine Transform) represented by JPEG and MPEG, and waveform coding, i.e., sub-band coding, wavelet coding, and fractal coding. Further, in order to remove redundant signals between images, inter-image prediction using motion compensation is performed, and a difference signal is subjected to waveform coding.
MPEG, which is based on motion compensation DCT, will be described.
An input image to be processed is divided into a plurality of 16xc3x9716 macroblocks. Each 16xc3x9716 macroblock is further divided into 8xc3x978 blocks. Each 8xc3x978 block is subjected to DCT. The resulting transform coefficients are quantized and then, transmitted or stored. This is called intra-frame coding.
On the other hand, using a method of detecting motion, such as block-matching which matches blocks, a predictive macroblock, which has the least error to a target macroblock of a frame, is detected among macroblocks of another frame temporally adjacent to the former frame. Motion is also detected. An optimal predictive block is obtained by subjecting a previous image to motion compensation based on the detected motion. A signal indicating a predictive macroblock having the least error is a motion vector. An image to be referred for generating a predictive macroblock is a reference image.
Next, a difference between a target block among the 8xc3x978 blocks into which the input image is divided, and its corresponding predictive block, is calculated. The difference is subjected to DCT, its transform coefficients are quantized, and the result is transmitted or stored along with motion information. This is inter-frame coding. At the receiver, the difference signal is restored from the quantized transform coefficients. Thereafter, a predictive block corresponding to the target block is calculated based on the motion vector transmitted or stored. The predictive block is added with the difference signal to reproduce the image.
MPEG2 has two motion compensation modes for an interlaced image, i.e., a frame motion compensation mode and a field motion compensation mode. In the frame motion compensation, the data of an odd-numbered field and an even-number field of a block to be decoded are calculated based on the same motion vector. In the field motion compensation mode, the data of an odd-numbered field and an even-number field are obtained from two different motion vectors. In interlaced scanning, an odd-number field and an even-numbered field are scanned at different moments, so there is sometimes a large difference in the amount of motion between both of the fields. For this reason, when such an interlaced image is subjected to motion compensation, if an odd-numbered field and an even-number field are subjected to different motion compensation, the error of a predictive signal is made small, and resolution in the temporal region can be maintained.
On the other hand, according to the standard H.263 established by ITU-T, as to motion compensation in decoding an image, a predictive block is generated in a way in which in addition to a motion vector of a target block to be decoded, motion vectors of blocks adjacent to the target block are used to produce predictive data, and the weighted average of the predicted data is calculated, which is a predictive block. This is called overlapped motion compensation. By the overlapped motion compensation, errors of a predictive signal to a target block and the predictive signals to all adjacent blocks are averaged, thereby suppressing the difference signal, and reducing distortion between blocks, i.e., a phenomenon where the border between two blocks is clearly perceivable, as observed in cases where motion compensation is performed block by block.
When the overlapped motion compensation is applied to an interlaced image, the error of a predictive signal is required to be small. However, the interlaced image is processed with the frame motion compensation mode and the field motion compensation mode, so that there is a case where to adjacent blocks are subjected to motion compensation with different modes. The frame motion compensation mode is effective for a static region in which a high-frequency component between scanning lines is small. The field motion compensation mode is suitable for a region having a large motion in which a high-frequency component between scanning lines is large. When a target block is processed with the frame motion compensation mode while the adjacent block is processed with the field motion compensation mode, because the predictive data calculated based on the motion vector of the adjacent block has a large high-frequency component between scanning lines, the overlapped motion compensation introduces a high-frequency component to the predictive signal of the target block. As opposed to this, when a target block is processed with the field motion compensation mode while the adjacent block is processed with the frame motion compensation mode, the averaging in the overlapped motion compensation reduces the temporal resolution of a predictive signal, thereby blurring a motion in the region having a large motion.
It is an object to provide an image predictive decoding method improving a coding efficiency by generating a highly efficient predictive signal when overlapped motion compensation is applied to an interlaced image, an apparatus for the method, and a data storage medium for the method.
Other objects and advantages of the present invention will become apparent from the detailed description desired hereinafter; it should be understood, however, that the detailed description and specific embodiment are desired by way of illustration only, since various changes and modifications within the scope of the invention will become apparent to those skilled in the art from this detailed description.
According to a first aspect of this invention, there is provided an image predictive decoding method including
a decoding method in which when decoding compressively coded image data resulting from predicting the data of a predictive region to a target region, using information about a plurality of motion compensation modes, and a plurality of motion vectors corresponding to the plurality of motion compensation modes, and performing coding, for a target region to be decoded, target predictive region data calculated based on the motion vector of the target region, and adjacent predictive region data calculated based on the motion vector of at least one adjacent region to the target region, are weightedly averaged to produce optimal predictive region data to the target region,
said image predictive decoding method calculating the adjacent predictive region data, which is calculated based on the motion vector of at least one adjacent region to the target region, based only on the motion vector of the adjacent region having the same motion compensation mode as the target region, to produce the optimal predictive region data.
Therefore, it is possible to suppress the error of a predictive signal by performing overlapped motion compensation to interlaced image while getting around the problems that the overlapped motion compensation introduces high-frequency components to a static region, and that the motion in a region having a large motion blurs due to a reduction in the temporal resolution of a predictive signal by averaging in the overlapped motion compensation.
According to a second aspect of this invention, there is provided an image predictive decoding method including
a decoding method in which when decoding compressively coded image data including information about a plurality of motion compensation modes, and a plurality of motion vectors corresponding to the plurality of motion compensation modes, for a target region to be decoded, target predictive region data calculated based on the motion vector of the target region, and adjacent predictive region data calculated based on the motion vector of at least one adjacent region to the target region, are weightedly averaged to produce optimal predictive region data to the target region,
said image predictive decoding method calculating the adjacent predictive region data, which is calculated based on the motion vector of at least one adjacent region to the target region, based on the motion vector of the adjacent region when the adjacent region has the same motion compensation as the target region, or based on the motion vector of the target region when the adjacent region has a motion compensation mode different from that of the target region, to produce the optimal predictive region data.
Therefore, it is possible to suppress the error of a predictive signal by performing overlapped motion compensation to interlaced image while getting around the problems that the overlapped motion compensation introduces high-frequency components to a static region, and that the motion in a region having a large motion blurs due to a reduction in the temporal resolution of a predictive signal by averaging in the overlapped motion compensation.
According to a third aspect of this invention, there is provided the image predictive decoding method of any of the first and second aspects includes, as the motion compensation mode,
a frame motion compensation mode in which the data of the even-numbered field and odd-numbered field of a predictive region are calculated based on the same motion vector, and
a field motion compensation mode in which the data of the even-numbered field of a predictive region is calculated based on a first field motion vector, and the data of the odd-numbered field of the predictive region is calculated based on a second field motion vector.
Therefore, the same effect as in the first and second aspects are obtained.
According to a fourth aspect of this invention, there is provided an image predictive decoding method including
a decoding method in which when decoding compressively coded image data resulting from predicting predictive region data to a target region, using information about a frame motion compensation mode in which the data of the even-numbered field and odd-numbered field of the predictive region are calculated based on the same motion vector, information about a field motion compensation mode in which the data of the even-numbered field of the predictive region is calculated based on a first field motion vector, and the data of the odd-numbered field of the predictive region are calculated based on a second field motion vector, and motion vectors corresponding to the plurality of motion compensation modes, and performing coding, for a target region to be decoded, target predictive region data calculated based on the motion vector of the target region, and adjacent predictive region data calculated based on the motion vector of at least one adjacent region to the target region, and weightedly averaged to produce optimal predictive region data to the target region.
said image predictive decoding method, when both a target region and the adjacent region are in the field motion compensation mode, calculating the data of an odd-numbered field of target predictive region based on the motion vector of the target region, the data of an even-numbered field of target predictive region based on the motion vector of the target region, the data of a first odd-numbered field of predictive region based on the motion vector of the adjacent region, the data of an even-numbered field of predictive region based on the motion vector of the adjacent region, weightedly averaging the data of an odd-numbered field of target predictive region and the data of a first odd-numbered field of predictive region, and the data of an even-numbered field of target predictive region and the data of a first even-numbered field of predictive region, to produce averaged odd-numbered field data and averaged even-numbered field data, respectively, and arranging the averaged odd-numbered field data and the averaged even-numbered field data in a frame structure, to produce optimal predictive region data.
Therefore, it is possible to suppress the error of a predictive signal by performing overlapped motion compensation to interlaced image when a target region and the adjacent region are in the same field predictive mode, while getting around the problems that the overlapped motion compensation introduces high-frequency components to a static region, and that the motion in a region having a large motion blurs due to a reduction in the temporal resolution of a predictive signal by averaging in the overlapped motion compensation.
According to a fifth aspect of this invention, there is provided an image predictive decoding method including
a decoding method in which when decoding compressively coded image data resulting from predicting predictive region data to a target region, using information about a frame motion compensation mode in which the data of the even-numbered field and odd-numbered field of the predictive region are calculated based on the same motion vector, information about a field motion compensation mode in which the data of the even-numbered field of the predictive region is calculated based on a first field motion vector, and the data of the odd-numbered field of the predictive region is calculated based on a second field motion vector, and motion vectors corresponding to the plurality of motion compensation modes, and performing coding, for a target region to be decoded, target predictive region data calculated based on the motion vector of the target region, and adjacent predictive region data calculated based on the motion vector of at least one adjacent region to the target region, are weightedly averaged to produce optimal predictive region data to the target region.
Said image predictive decoding method, when the target region is in the frame motion compensation mode while the adjacent region is in the field motion compensation, calculating the adjacent predictive region data, which is calculated based on the motion vector of at least one adjacent region to the target region, based on an averaged frame motion vector of the adjacent region resulting from averaging the first field motion vector of the adjacent region and the second field motion vector of the adjacent region.
Therefore, even when a region in the frame motion compensation mode and a region in the field motion compensation mode are adjacent to each other, it is possible to suppress the error of a predictive signal by adaptively changing methods of overlapping, depending on the motion compensation mode of the target region, without introducing high-frequency components to a region in the frame predictive mode, and reducing the temporal resolution of a region in the field predictive mode. Further, it is possible to suppress the error of a predictive signal by performing overlapped motion compensation to interlaced image while getting around the problems that the overlapped motion compensation introduces high-frequency components to a static region, and that the motion in a region having a large motion blurs due to a reduction in the temporal resolution of a predictive signal by averaging in the overlapped motion compensation.
According to a sixth aspect of this invention, there is provided an image predictive decoding method including
a decoding method in which when decoding compressively coded image data resulting from predicting predictive region data to a target region, using information about a frame motion compensation mode in which the data of the even-numbered field and odd-numbered field of the predictive region are calculated based on the same motion vector, information about a field motion compensation mode in which the data of the even numbered field of the predictive region is calculated based on a first field motion vector, and the data of the odd-numbered field of the predictive region is calculated based on a second field motion vector, and motion vectors corresponding to the plurality of motion compensation modes, and coding the predictive region data, for a target region to be decoded, target predictive region data calculated based on the motion vector of the target region, and adjacent predictive region data calculated based on the motion vector of at least one adjacent region to the target region, are weightedly averaged to produce optimal predictive region data to the target region.
said image predictive decoding method, when the target region is in the frame motion compensation mode while the adjacent region is in the field motion compensation, calculating the adjacent predictive region data, which is calculated based on the motion vector of at least one adjacent region to the target region, based on one of the first field motion vector of the adjacent region and the second field motion vector of the adjacent region, which has the smaller difference from the motion vector of the target region.
Therefore, even when a region in the frame motion compensation mode and a region in the field motion compensation mode are adjacent to each other, it is possible to suppress the error of a predictive signal by adaptively changing methods of overlapping, depending on the motion compensation mode of the target region, without introducing high-frequency components to a region in the frame predictive mode, and reducing the temporal resolution of a region in the field predictive mode. Further, when a target block is in the frame predictive mode while the adjacent region is in the field predictive mode, it is possible to suppress the error of a predictive signal by using the field motion vector of the adjacent block close to the frame motion vector of the target block, while getting around the problems that the overlapped motion compensation introduces high-frequency components to a static region, and that the motion in a region having a large motion blurs due to a reduction in the temporal resolution of a predictive signal by averaging in the overlapped motion compensation.
According to a seventh aspect of this invention, there is provided an image predictive decoding method including
a decoding method in which when decoding compressively coded image data resulting from predicting predictive region data to a target region, using information about a frame motion compensation mode in which the data of the even-numbered field and odd-numbered field of the predictive region are calculated based on the same motion vector, information about a field motion compensation mode in which the predictive data of the even-numbered field of the predictive region is calculated based on a first field motion vector, and the predictive data of the odd-numbered field of the predictive region is calculated based on a second field motion vector, and motion vectors corresponding to the plurality of motion compensation modes, and performing coding, for a target region to be decoded, target predictive region data calculated based on the motion vector of the target region, and adjacent predictive region data calculated based on the motion vector of at least one adjacent region to the target region, are weightedly averaged to produce optimal predictive region data to the target region.
said image predictive decoding method, when the target region is in the field motion compensation mode while the adjacent region is in the frame motion compensation calculating a second field motion vector of the adjacent region, based on a first field motion vector of the target region, a second field motion vector of the target region, and a first field motion vector of the adjacent region equal to the frame motion vector of the adjacent region in assuming that the adjacent region has the information about a field motion compensation mode in which the predictive data of the even-numbered field of the predictive region are calculated based on a first field motion vector, and the predictive data of the odd-numbered field of the predictive region are calculated based on a second field motion vector, and producing the adjacent predictive region data using the first- and second field motion vectors of the adjacent region.
Therefore, even when a target region and the adjacent region are in the different predictive modes, it is possible to suppress the error of a predictive signal by performing overlapped motion compensation to interlaced image while getting around the problems that the overlapped motion compensation introduces high-frequency components to a static region, and that the motion in a region having a large motion blurs due to a reduction in the temporal resolution of a predictive signal by averaging in the overlapped motion compensation.
According to an eighth aspect of this invention, there is provided an image predictive decoding method including
a decoding method in which when decoding compressively coded image data resulting from predicting the data of a predictive region to a target region, using information about a plurality of motion compensation modes, and a plurality of motion vectors corresponding to the plurality of motion compensation modes, and performing coding for a target region to be decoded, target predictive region data calculated based on the motion vector of the target region, and adjacent predictive region data calculated based on the motion vector of at least one adjacent region to the target region, are weightedly averaged to produce optimal predictive region data to the target region,
said image predictive decoding method calculating the adjacent predictive region data, when the target region is in the frame motion compensation mode, weightedly averaging the target predictive region data and the adjacent predictive region data, to produce the optimal predictive region data to the target region,
while when the target region is in the field motion compensation mode, the optimal predictive region data to the target region is calculated based only on the motion vector of the target region.
Therefore, it is possible to suppress the error of a predictive signal to performing overlapped motion compensation to interlaced image while getting around the problems that the overlapped motion compensation introduces high-frequency components to a static region, and that the motion in a region having a large motion blurs due to a reduction in the temporal resolution of a predictive signal by averaging in the overlapped motion compensation.
According to a ninth aspect of this invention, there is provided the image predictive decoding method of the eighth aspect wherein when the target region is in the frame motion compensation mode, optimal predictive region data to the target region is calculated in a way in which it is decided whether each adjacent region to the target region is in the field motion compensation mode, or not, and when the adjacent region is in the field motion compensation mode, the first and second motion vectors of the adjacent region are averaged, predictive region data to the adjacent region is calculated based on the averaged field motion vector of the adjacent region, and the target predictive region data and all of the adjacent predictive region data are averaged to produce the optimal predictive data to the target region.
Therefore, the same effect as in the eighth aspect is obtained.
According to a tenth aspect of this invention, there is provided an image predictive decoding apparatus comprising:
a frame memory;
an input unit for receiving compressively coded image data including information about a plurality of motion compensation modes, and a plurality of motion vectors corresponding to the plurality of motion compensation modes;
a data analyzer for analyzing the compressively coded image data, and outputting at least the information about the plurality of motion compensation modes, the plurality of motion vectors corresponding to the plurality of motion compensation modes, and image transform coefficients;
a decoder for restoring the image transform coefficients with a prescribed method to a decompressed difference image;
a predictive image generator for generating an optimal predictive image based on a reference image stored said frame memory; and
an adder for adding the decompressed difference image and the optimal predictive image to produce a reproduced image, outputting the same, and storing the same in said frame memory,
said predictive image generator producing optimal predictive region data using the image predictive decoding method of any of the first aspect to the ninth aspect.
Therefore, it is possible to provide an apparatus realizing an image predictive decoding method which gets around the problems that the overlapped motion compensation introduces high-frequency components to a static region, and that the motion in a region having a large motion blurs due to a reduction in the temporal resolution of a predictive signal by averaging in the overlapped motion compensation.
According to an eleventh aspect of this invention, there is provided an image predictive coding apparatus comprising:
a frame memory;
an output unit;
an input unit for receiving a digital image signal of each frame;
a predictive data generator for generating the data of a predictive block based on a reference image stored in said frame memory;
a first adder for outputting difference data between the data of a target block and the data of the predictive block, as the data of a residual block;
an encoder for compressing the data of the residual block, and outputting the compressed data of the residual block;
a variable length encoder for subjecting the compressed data of the residual block to variable length coding, and outputting the resulting coded signal through said output unit;
a decoder for decompressing the compressed data of the residual block, and outputting the decompressed data of the residual block; and
a second adder for adding the decompressed data of the residual block to the data of the predictive block, outputting the data of a resulting reproduced block, and storing the data of the reproduced block in said frame memory;
said predictive data generator performing predictive data production corresponding to the optimal predictive region data production in the image predictive decoding method of any of the first aspect to the ninth aspect.
Therefore, it is possible to provide an apparatus performing encoding for an image predicting decoding method which gets around the problems that the overlapped motion compensation introduces high-frequency components to a static region, and that the motion in a region having a large motion blurs due to a reduction in the temporal resolution of a predictive signal by averaging in the overlapped motion compensation.
According to a twelfth aspect of this invention, there is provided a data storage medium storing a program for performing the image predictive decoding method of any of the first to ninth aspects, using a computer.
Therefore, it is possible to provide a storage medium containing a program executing an image predictive decoding method which gets around the problems that overlapped motion compensation introduces high-frequency components to a static region, and that the motion in a region having a large motion blurs due to a reduction in the temporal resolution of a predictive signal by averaging in the overlapped motion compensation.